Optimal placebo-treatment comparisons in trials with an incomplete within-subject design and heterogeneous costs and variances

The aim of a clinical trial is to compare placebo to one or more treatments. The within-subject design is known to be more efficient than the between-subject design. However, in some trials that implement a within-subject design it is not possible to evaluate the placebo and all treatments within each subject. The design then becomes an incomplete within-subject design. An important question is how many subjects should be allocated to each combination of placebo and treatments. This paper studies optimal allocations of subjects in trials with a placebo and two treatments under heterogenous costs and variances. Two optimality criteria that consider the placebo-treatment contrasts simultaneously are considered, and the design is derived under a budgetary constraint. More subjects are allocated to those combinations with higher variances and lower costs. The optimal allocation is compared to the uniform allocation, which allocates equal number of subjects to each placebo and treatment combination, and to the complete within-subject design, where placebo and all treatments are available in each subject. The methodology is illustrated on the basis of an example on consultation time in primary care. A Shiny app is available to facilitate use of the methodology.


Introduction
The randomized controlled trial is the gold standard for the comparison of multiple experimental conditions [1,2]. An important question in the design phase of such a trial is how many subjects should be assigned to each of these conditions. Often a uniform design is chosen, so that an equal number of subjects is assigned to each condition. A rationale for the uniform design is that it is consistent with the view of clinical equipoise that must exist before the start of the trial [3]. However, the uniform design is not necessarily an efficient choice, especially so when costs and/or variances differ across the conditions [4]. A more efficient design is achieved when the conditions with highest variance and lowest costs are assigned more subjects. The past twenty years various papers appeared on the optimal allocation of subjects to experimental conditions: for the comparison of two conditions on a quantitative outcome [5][6][7] or a categorical outcome [8][9][10][11][12][13] and for the comparison of more than two conditions on a quantitative outcome [3,[14][15][16][17][18][19][20][21] or a categorical outcome [22][23][24]. The main focus of the existing literature is on trials with a between-subject design, where each subject receives only one condition. Such a design is also referred to as the parallel group design. However, in some trials it is also possible to implement a within-subject design, so that each subject receives at least two conditions sequentially. Such a design is also referred to as a crossover design: subjects cross over from the one condition to the during the course of the trial [25,26]. Because multiple conditions are available within each subject, each subject serves as its own control and hence the design is more efficient than the between-subject design.
In the ideal case each of the conditions is available within each subject so that the design is a complete within-subject design. However, in practice this is not always possible to achieve, especially so in trials with a limited study duration, a long onset time of treatment and/or a long wash-out period between subsequent conditions. Other reasons to limit the number of conditions per subject are participation burden and hence attrition, ethical concerns and the need to avoid test-retest effects. The latter occur when the same measurement instrument, such as a cognitive ability test or a behavioral test, is used too often in a limited amount of time. Consider as an example a pharmaceutical trial that compares different drugs to a placebo for children with attention deficit hyper activity disorder. Depending on the onset time of each drug, the number of different drugs than can be evaluated per child will be small in a trial of limited duration, especially so if washout periods are needed to avoid carry-over effects of a drug to the next period(s). In addition to that, too many drug switches per child may not be acceptable from an ethical point of view. Another example is a trial that compares the performance of different types of new and innovative sport shoes for professional marathon runners to a standard type. Runners are advised to limit the number of marathons per year and to take a few weeks or even months to recover before the next marathon. For that reason the number of type of sport shoes that can be evaluated per runner in a study of limited duration is rather small. In both examples the number of conditions per subject is lower than the actual number of conditions that is to be evaluated in the trail, hence the design is an incomplete within-subject design.
The aim of this contribution is to further study the optimal allocation of subjects for placebo-treatment comparisons under a budgetary constraint. The outcome is quantitative and the variances and costs are allowed to be heterogeneous. Attention is limited to trials with one placebo and two treatments. The optimal allocation of subjects for the incomplete within-subject design is compared to the uniform allocation. As such, the loss of efficiency that results from assigning equal number of subjects to each combination of experimental conditions can be quantified. Furthermore, the incomplete within-subject design is compared to the complete within-subject design. As such, the loss of efficiency from not having each experimental within each subject is quantified.
The optimal allocation of subjects to conditions depends on various factors: a budgetary constraint, the optimality criterion and the covariance matrix between the outcomes per subject. The budgetary constraint limits the number of subjects that can be included in the study because the costs of a study should not exceed its budget. Costs consist of, among others, overhead costs per subject and costs per subject per placebo and treatment. There exist various allocations of subjects for which the costs are less than or equal to the budget. An optimality criterion is therefore needed to select the optimal allocation. In this contribution the focus is on placebo-treatment comparisons. As there are two treatments, there are also two placebotreatment comparisons. The allocation that is optimal for the comparison of placebo and treatment 1 may be far from optimal for the comparison of placebo to treatment 2. It is therefore necessary to use optimality criteria that consider both placebo-treatment comparisons simultaneously. The optimal allocation further depends on the variances and covariances between the outcomes within a subject. This contribution considers a general covariance matrix with heterogeneous variances and covariances. Once the costs and budget are specified, an a priori estimate of the covariance matrix is provided and the optimality criterion has been chosen, the optimal allocation of subjects can be derived. This paper builds upon previous research on optimal designs for the crossover trial with a budgetary constraint [27][28][29]. In these contributions two treatment conditions are taken into account and each subject receives both conditions. In the current paper three conditions are taken into account, but each subject receives only two out of these three conditions. This contribution is organized as follows. The next section formulates the statistical model, shows how the means in the placebo and treatments are estimated along with the associated covariance matrix and introduces the optimal design methodology. Two optimality criteria are specified along with the budgetary constraint and it is explained how the optimal allocation to treatment combinations is derived numerically. The results section gives optimal allocations for the incomplete within-subject design and compares the optimal allocations to the uniform design and to the complete within-subject design on the basis of their efficiencies. An illustrative example on consultation time in primary care illustrates the use of the optimal design methodology. Conclusions, discussion and directions for future research are given in the final section.

Statistical model
In the within-subject design the data have a multilevel structure with multiple outcome measures nested within subjects. Such data can be analyzed using the multilevel model [30,31], which is also known as the mixed model or hierarchical model [32].
We first focus on the model for the complete within-subject design, where each subject receives the placebo and both treatments. Subsequently it is shown how this model can be adjusted to account for the incomplete within-subject design. At the end of each treatment, outcome measurements are taken on each subject. The model for quantitative outcome y ij for experimental condition i (i = 0 for placebo and i = 1, 2 for treatments 1 and 2, respectively) in subject j = 1,. . .N is given by Here, μ i is the mean score for condition i, and e ij is the residual. For the latter we assume Eðe ij Þ ¼ 0; varðe ij Þ ¼ s 2 i and covðe ij ; e i0j Þ ¼ s ii0 . As such, the model allows for heterogenous variances and covariances. It should be noted that this model does not account for temporal effects, such as models for crossover trials often do.
The model can be written in matrix-vector notation: The vector of outcomes of subject j is The vector of placebo and treatment means is with corresponding design matrix In the complete within-subject design the design matrix is the identity matrix. The vector of residuals is of subject j is: with corresponding design matrix Again the design matrix is the identity matrix in the complete within-subject design. The 3×3 covariance matrix of the residuals e j is D ¼ The covariance matrix of the response vector y j is calculated as The vector of placebo and treatment means is estimated aŝ with corresponding covariance matrix In the incomplete within-subject design certain entries should be removed from the vectors with outcomes and residuals and certain rows should be removed from the design matrices. If the placebo is not available within subject j, then the first entry should be removed from the vectors y j and e j and the first row should be removed from the design matrices X j and Z j . If treatment i is not available within subject j, then entry i+1 should be removed from the vectors y j and e j and row i+1 should be removed from the design matrices X j and Z j . Eqs (9)-(11) can then be used to estimate the placebo and treatment means and the associated covariance matrix.

Optimal design methodology
Optimality criteria. To derive the optimal allocation of subjects to treatment combinations an optimality criterion is needed. The focus is on placebo-treatment comparisons, of which there are two in a trial with two treatments. For both of them a contrast a T 0i μ can be formulated, with a T 01 ¼ ð1; À 1; 0Þ and a T 02 ¼ ð1; 0; À 1Þ. The variance of these contrasts is calculated as The optimal allocation is the one that minimizes the contrast variance F 0i . The allocation that is optimal for the first contrast may be far from optimal for the second. For that reason the contrast variances F 0i should be considered simultaneously. This will be done by implementing a D A -optimal design [33,34] or a compound optimal design [35].
For the D A -optimal design both vectors of contrast coefficients are stacked in one matrix: The optimality criterion to be minimized is then As such, the optimal allocation minimizes the volume of the confidence ellipsoid of the contrasts.
For the compound optimal design a linear combination of the contrast variances F 0i is minimized: where 0�λ 1 �1. The user-specified weight λ 1 reflects the relative importance of the first contrast in the compound optimal design. This criterion was used previously in a between-subject design where costs and variances were equal across placebo and treatments [14]. The optimal design may be compared to other designs that are of interest, for instance the uniform design. This may be done by means of the relative efficiency. For the D A -optimal design the relative efficiency of an allocation ξ versus the optimal allocation ξ* is The parts in square brackets of the numerator and denominator are the value of the optimality criterion F D A as found for the allocations ξ* and ξ, respectively. Note that both the numerator and denominator are taken to the power ½, since two contrasts are taken into account. For the compound optimal design the relative efficiency is calculated as The numerator and denominator are the value of the optimality criterion F compound as found for the allocations ξ* and ξ, respectively.
The relative efficiency lies between 0 and 1 and its reciprocal quantifies how many times a trial with allocation ξ should be replicated to perform as well as the optimal allocation ξ*. A trial with RE = 0.8 need to be replicated 1.25 times and a trail with RE = 0.9 need to be replicated 1.11 times. Stated differently, a trail with RE = 0.8 needs an increase in sample size of Budgetary constraint. As is obvious, any design becomes more efficient when sample size increases. In practice, however, sample size cannot increase within bounds due to financial limitations. A budgetary constraint is therefore used to derive the optimal allocation to treatment combinations.
The overhead costs per subject are denoted C s and consist of costs such as administrative costs, incentives and insurances. These costs do not depend on the number and order of the conditions that are assigned to a subject. The costs for placebo per subject are denoted C 0 and the costs for treatment i per subject are denoted C i . The budget that is available is denoted B.
The costs per subject depend on which condition(s) the subject receives. In the incomplete within-subject design the costs per subject are C s01 = C s +C 0 +C 1 for a subject who receives placebo combined with treatment 1, C s02 = C s +C 0 +C 2 for a subject who receives placebo combined with treatment 2 and C s12 = C s +C 1 +C 2 for a subject who receives treatment 1 combined with treatment 2. The corresponding numbers of subjects are denoted N 01 , N 02 and N 12 , respectively. The costs are then C s01 N 01 +C s02 N 02 +C s12 N 12 . The design space is O = (N 01 �0, N 02 �0, N 12 �0|costs�B), with N 01 , N 02 and N 12 integers. The optimal allocation ξ* is the one for which the optimality criterion F D A or F compound is minimized.
In the complete within-subject design the costs per subject are C s012 = C s +C 0 +C 1 +C 2 . The number of subjects that can be included are N 012 = B/C s012 . As all subjects receive placebo combined with treatments 1 and 2, it is not necessary to derive the optimal allocation of subjects.
Derivation of optimal allocation. Simple mathematical expressions for the optimal allocation cannot be derived. For that reason the optimal allocation is found numerically. For optimization problems like this it is straightforward to use an optimizer for functions with constraints, such as the optimizer constrOptim.nl in the R [36] package alabama [37]. In this optimization problem the function is the variance of the treatment effect estimator (with the sample sizes as parameters), and the constraints are the cost constraint and the constraint that all sample sizes are natural numbers (i.e. integers of zero or larger). Unfortunately, the optimizer constrOptim.nl appeared to be very sensitive to the starting values of the iterative optimization process. Also, the solution by default produced non-integer sample sizes. For that reason another approach to find the optimal allocation was used: the variance of the treatment effect estimator is calculated for all possible combinations of integer samples for the which the cost constraint holds. The combination that results in the smallest variance is the optimal allocation. The amount of time this approach requires depends on the number of combinations of sample sizes, which in its turn depends on the costs and budget. For instance, the optimal allocations in Table 1 in the next section took a few seconds each to calculate, and those in Table 2 took about one minute each to calculate using a computer with 16 GB RAM and a processor with 2.8 GHz.

Results
The following subsections present optimal allocations for three different combinations of costs per subject for placebo and treatments 1 and 2. The first is C 0 = 10, C 1 = 20, C 2 = 20 so that the costs of both treatments are equal to one another and twice as large as those for placebo. The second is C 0 = 10, C 1 = 20, C 2 = 30 so that the costs of treatment 1 are twice as large as those of placebo and those of treatment 2 are one and a half times as large as those of treatment 1. The third is C 0 = 10, C 1 = 20, C 2 = 40 so that the costs of treatment 1 are twice as large as those of placebo and those of treatment 2 are twice as large as those of treatment 1. The overhead costs per subject are C S = 100 and the budget is B = 50000.  compound optimal design with λ 1 = 0.25 141  243  384  138  229  367  133 218 351 123  261  384  124  242  366 118 231 349 273  384  110  255  365  103  244  347 compound optimal design with λ 1 = 0.5 compound optimal design with λ 1 = 0.75  compound optimal design with λ 1 = 0.25 Here the variance in treatments 1 and 2 are twice and four times as large as in placebo, respectively. For the complete and incomplete within-subject design all pairwise correlations are set to ρ ii 0 = 0.5, so that they are of large size according to Cohen [38].
For each combination of costs and covariance matrix the D A -optimal design and compound optimal design are derived. For the latter three different values of the weight λ 1 are used: λ 1 = 0.25, 0.5 and 0.75. For the first value of λ 1 the main focus is on the comparison of placebo with treatment 2, for the second both placebo-treatment comparisons are of equal importance and for the third the main focus is on the comparison of placebo with treatment 1.

Incomplete within-subject design
For this design there are two conditions per subject. Two different scenarios are considered. In scenario 1 subjects are allocated to receive either placebo with treatment 1 or placebo with treatment 2. In scenario 2 subjects are allocated to receive either placebo with treatment 1, placebo with treatment 2, or treatment 1 with treatment 2. In scenario 1 the combination of both treatments 1 and 2 is not taken into account. This scenario may be chosen when it is considered unethical to allocate both treatments to a subject, for instance when both are expected to have harmful side effects. Table 1 presents the optimal sample sizes N * 01 ; N * 02 and their sum N* for scenario 1. We first discuss results for the compound optimal designs. For any combination of costs C 0 , C 1 , C 2 and weight λ 1 the optimal sample size N * 01 decreases, and hence the optimal sample size N * 02 increases, when the ratio s 2 2 =s 2 1 becomes larger. Fewer subjects receive placebo combined with treatment 1, and hence more receive placebo combined with treatment 2, if the variance of outcomes in treatment 2 increases relative to the variance of the outcomes in treatment 1. This confirms results previously found for a between-subject comparison of two means: more subjects are assigned to the treatment with highest variance.
For those designs with equal costs C 1 = C 2 = 20 in treatments 1 and 2, the total sample size remains constant at N* = 384, irrespective of the chosen variances and weight λ 1 . This can be explained as follows: as both treatment combinations are equally expensive, a decrease of the optimal sample size N * 01 by a certain amount results in an increase of the optimal sample size N * 02 by the same amount. For unequal costs (C 1 , C 2 ) = (20,30) and (20,40) this result does not hold: increasing the ratio s 2 2 =s 2 1 results in lower N * 01 , higher N * 02 but lower N*. For any combination of variance ratio s 2 2 =s 2 1 and weight λ 1 the optimal sample size N * 02 and total sample size N* decrease if the costs C 2 for treatment 2 increase. Fewer subjects are assigned to placebo combined with treatment 2 if the costs for treatment 2 increase and consequently the total sample size decreases as well. The relationship between C 2 and N * 01 is not always monotone.
The optimal sample sizes further depend on the weight λ 1 of the compound optimal design. For any combination of costs C 0 , C 1 , C 2 and variance ratio s 2 2 =s 2 1 the optimal sample size N * 01 increases, and hence the optimal sample size N * 02 decreases, if the weight λ 1 increases. This is also obvious: when more emphasis is put on the comparison of placebo to treatment 1 rather than to treatment 2, more subjects will be allocated to receive placebo combined with treatment 1 than placebo combined with treatment 2.
Only when variances and costs are equal for treatments 1 and 2 and when there is equal interest in both placebo-treatment comparisons (i.e. λ 1 = 0.5) the optimal allocation is the uniform allocation. In all other cases the two optimal sample sizes N * 01 and N * 02 are different from each other. In the most extreme cases in Table 1 N * 02 ¼ 0:5N * 01 and N * 02 ¼ 2:5N * 01 . We now focus on results for the D A -optimal design. The optimal sample sizes N * 01 ; N * 02 and their sum N* are not at all or only slightly influenced by the variance ratio s 2 2 =s 2 1 . Increasing the costs C 2 of treatment 2 hardly influences the optimal sample size N * 01 , but results in a decrease of the optimal sample size N * 02 and total sample size N*. Fewer subjects receive placebo combined with treatment 2 if the costs for treatment 2 increase. As a result of that, total sample size also becomes smaller. Table 2 presents the optimal sample sizes N * 01 ; N * 02 ; N * 12 and their sum N* for scenario 2. Again, results for the compound optimal designs are discussed first. For any combination of costs C 0 , C 1 , C 2 and weight λ 1 it can be observed that increasing the variance ratio s 2 2 =s 2 1 results in decreasing optimal sample size N * 01 , decreasing and subsequently increasing optimal sample size N * 02 , increasing optimal sample size N * 12 and decreasing total sample size N*. The effects of s 2 2 =s 2 1 on N * 02 and N* are not as pronounced as those on N * 01 and N * 12 . For any combination of variance ratio s 2 2 =s 2 1 and weight λ 1 the optimal sample sizes N * 02 ; N * 12 and total sample size N* decrease with increasing costs C 2 . As is obvious, fewer subjects are assigned to receive treatment 2, either combined with placebo or treatment 1, if the costs of treatment 2 increase. The relationship between N * 01 and s 2 2 =s 2 1 is not always monotone. For any combination of costs C 0 , C 1 , C 2 and variance ratio s 2 2 =s 2 1 the optimal sample size N * 01 increases while the optimal sample size N * 02 decreases if the weight λ 1 increases. This is obvious since more subjects are assigned to receive placebo combined with treatment 1 rather than placebo combined with treatment 2 if comparing placebo to treatment 1 becomes of more interest. The relation between λ 1 and N * 12 is not always monotone. We now focus on the optimal allocation for the D A -optimality criterion. The optimal sample sizes N * 01 ; N * 02 and total sample size N* decrease with increasing variance ratio s 2 2 =s 2 1 , whereas the optimal sample size N * 12 increases. Furthermore, the optimal sample size N * 01 increases with increasing costs C 2 whereas the optimal sample sizes N * 02 and N * 12 and total sample size N* decrease.

Complete within-subject design
For this design all subjects receive placebo combined with treatments 1 and 2 and it is therefore not necessary to derive an optimal allocation. As was already explained, the number of subjects is N 012 = B/C s012 . In other words: the number of subjects is as large as the budget allows. It can be calculated that N 012 = 333 subjects can be included when C 0 = 10, C 1 = 20, C 2 = 20, N 012 = 312 when C 0 = 10, C 1 = 20, C 2 = 30 and N 012 = 294 when C 0 = 10, C 1 = 20, C 2 = 30. Table 3 presents the efficiency of the uniform allocation relative to the optimal one. These efficiencies are equal or almost equal to 1 for the D A -optimality criterion, so the uniform design performs extremely well for this criterion. For the compound optimal designs with λ 1 = 0.5 all relative efficiencies are above 0.9. In the case of equal interest in both placebo-treatment comparisons the uniform design is highly efficient. For the compound optimal designs with λ 1 = 0.75 only three out of the 18 relative efficiencies are below 0.9. The lowest one is 0.84, meaning the uniform allocation needs an increase in total sample size of 20% to perform as well as the optimal allocation. For the compound optimal designs with λ 1 = 0.25 11 out of 18 of the relative efficiencies are below 0.9 but all of them are still above 0.85.

Efficiency of the incomplete within-subject design
A comparison of the total sample sizes in Tables 1 and 2 and those calculated for the complete within-subject design show that the total sample size is smallest for the complete-within subject design, followed by the incomplete within-subject design with scenario 1, and the incomplete within-subject design with scenario 2. The fewer experimental conditions are available within a subject, the more subjects can be included in the trial. However, a larger number of subjects does not imply a larger design efficiency. This follows from Table 4, which presents the efficiency of the incomplete within-subject designs relative to the complete within-subject design. The complete within-subject design is the most efficient, followed by the incomplete within-subject design with scenario 2 and with scenario 1, respectively. The more experimental conditions are available within a subject, the more efficient the design. Allowing for the combination of treatment 1 with 2 results in a more efficient design.

Illustrative example: Consultation time in primary care
The average consultation time in primary care in the Netherlands is only 10 minutes [39]. This short amount of time may result in too high work pressure, dissatisfaction, stress and burnout of the general practitioner. Suppose a trial is set up to study the effects of longer consultation time. There are three experimental conditions: the standard consultation time of 10 minutes, consultation time of 15 minutes and consultation time of 20 minutes. The time period of data collection is limited so that only two conditions can be evaluated per general practitioner, and one week is available per condition. Furthermore, each general practitioner should evaluate the standard consultation time. This results in two combinations of consultation time: combination 1 has 10 minutes combined with 15 minutes, and combination 2 has 10 minutes combined with 20 minutes. compound optimal design with λ 1 = 0.25 Note: B = 50,000 and C s = 100 and ρ ii 0 = 0.5.
RE W1 relative efficiency of uniform versus optimal allocation for the within-subject design with scenario 1.
RE W2 relative efficiency of uniform versus optimal allocation for the within-subject design with scenario 2.
The costs of a consult are 10.59 euros per ten minutes. Suppose a general practitioner spends 20 hours per week on consults. Then he or she can do 120 consults of 10 minutes a week, 80 consults of 15 minutes a week and 60 consults of 20 minutes a week. To compensate for the decrease in the number of consults that can be done per week, general practitioners in combination 1 receive C 1 = 40*10.59 = 423.6 euros and those in combination 2 receive C 2 = 60*10.59 = 635.4 euros to hire a colleague. On top of that, they all receive incentives of C s = 500 euros per practitioner. Furthermore, C 0 = 0, meaning that no compensation is given for the week in which the standard consultation time is used. From these costs is follows C s01 = 923.6 and C s02 = 1135.4. The budget is B = 100,000 euros.
To derive the optimal allocation a priori estimates of the variances and covariances are needed. The outcome variable is stress level, which is measured at the end of the two weeks using an instrument that ranges from 0 to 100. Relevant a priori estimates of the variances and covariances were unfortunately not available in the literature. In such a case one could consult an expert to get plausible estimates. Let us assume the anticipated variances are s 2 0 ¼ 100 for 10 minutes consultation time,  consultation time. In that case N * 01 ¼ 108 and N * 02 ¼ 0. Furthermore Fig 1 shows that the total sample size increases from N* = 88 for λ 1 = 0 to N* = 108 for λ 1 = 1.
Suppose more interest lies in the comparison of 10 with 15 minutes than that of 10 minutes with 20 minutes. This goal may be achieved by choosing for instance λ 1 = 0.75. The optimal sample sizes are then N * 01 ¼ 64; N * 02 ¼ 36, with total sample size N* = 100 and costs N * 01 C s01 þ N * 02 C s02 ¼ 99984:80 euros. The optimal allocation assigns more general practitioners to the combination with 10 and 15 minutes consultation time than to the combination with 10 and 20 minutes. This is obvious since as there is more interest in the comparison of 10 minutes to 15 minutes than in the comparison of 10 minutes to 20 minutes, more general practitioners are assigned to that first combination of consultation times. On top of that, C s01 is larger than C s02 and the optimal allocation assigns more general practitioners to the combination with lower costs. The optimal allocation is more efficient than the uniform allocation, which has N 01 = N 02 = 48, but the uniform allocation is highly efficient with a relative efficiency of 0.923.
Given the optimal allocation, one can calculate which effects sizes can be detected for both contrasts at the available budget in a one-sided test with power level 0.8 and type I error rate 0.05. The effect size is calculated as ðz 1À a þ z 1À b Þ � ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi varðm 0 À m i Þ p , where z 1−α and z 1−β are the 100(1−α)th and 100(1−β)th percentiles of the standard normal distribution. The variances of the contrasts are var(μ 0 −μ 1 ) = 2.2 and var(μ 0 −μ 2 ) = 4.2. The effect sizes are then calculated as 3.7 and 5.1 for the first and second contrast, respectively. This corresponds to standardized effect sizes as expressed by Cohen's d of 0.35 and 0.46, respectively. So the minimum effects that can be detected in this trial are small (d = 0.2) to medium (d = 0.5) in size.
One can also calculate the power to detect a user-specified effect size for the optimal allocation. The power to detect a medium effect size is 0.97 for the first contrast and 0.86 for the second. Much lower power levels of 0.41 and 0.29 are achieved when a small effect size is to be detected.

Discussion
The within-subject design is more efficient than the between-subject design. However, it is not always possible to evaluate the placebo and all treatments in each subject. In such a case the design is an incomplete-within subject design. This contribution studied optimal allocation of subjects to combinations of placebo with two treatments. The costs and variances were allowed to vary across placebo and treatments. Two optimality criteria that simultaneously take into account both placebo-treatment comparisons were considered and the design was sought under a budgetary constraint. In general it can be stated that more subjects are allocated to combinations of treatments that have high variance and low costs. For the compound optimal design the optimal allocation further depends on the weight λ 1 . When λ 1 increases then there is more interest in the comparison of the placebo with treatment 1 rather than with treatment 2, and hence more subjects are allocated to combinations that include treatment 1. Tables 1 and 2 give an overview of optimal allocations in a trial with a budget of 100000 for selected values of the costs and variances. Other combinations of the budget, costs and variances will, of course, result in different optimal allocations. A researcher who is planning to a trail should provide a priori estimates of the variances and covariances, along with the budget and costs and calculate the optimal allocation using the Shiny app. A priori estimates of the variances and covariances may be obtained from the literature or expert knowledge. Of course, there is no guarantee such estimates may hold in a different year, country or setting. It is therefore advised to do a robustness analysis. The optimal allocation may be derived for various plausible values of the variances and covariances. The allocation is robust if it hardly depends on these values. Otherwise, one may want to use robust optimal design techniques, such as an internal pilot or a maximin optimal design. This contribution restricted to trials with one placebo and two treatments. In the incomplete-within-subject design three combinations could be distinguished: placebo with treatment 1, placebo with treatment 2 and treatment 1 with treatment 2. Of course it is possible to apply the optimal design methodology to trials with more than two treatments. It should be taken into account that more treatments imply more combinations of placebo and treatments and hence deriving the optimal allocation becomes more computationally demanding. Consider a trial with a placebo and three treatments. There are six combinations in the case two experimental conditions can be evaluated in each subject and another four combinations if three experimental conditions can be evaluated in each subject. These numbers reduce to three and three in the case the placebo should be evaluated in every subject. For four and more treatments the number of combinations even further increases. Finding the optimal allocation may then become very time-consuming.
The within-subject design is a longitudinal study design in which two or more measures on the same outcome are taken over time on each subject. Longitudinal studies are very often hampered by attrition, meaning that subjects leave the study prematurely. Attrition may occur when subjects lose interest in the study or experience severe side-effects from one treatment and are not willing to continue with the next treatment. Attrition results in a less efficient design by default and it should be avoided at all costs. However, this is often easier said than done in practice. As an alternative, one may derive optimal allocations that take into account anticipated rates of attrition. The number of subjects of such a trial is larger than in a trial without attrition so that the available budget is still efficiently used.
Another important direction for future research are optimal allocations in the case the outcome variable is categorical, such as the prevalence of some kind of behavior (i.e. binary outcome) or the severity of such behavior (i.e. ordinal outcome). It may also be of interest to study optimal allocations when subjects are nested within health professionals, such as clinical child psychologists in the first example of the introduction and sport coaches in the second example. Such health professionals may differ with respect to their skills, experience, empathy, motivation and enthusiasm, and such differences should be taken into account in the multilevel model by adding an extra level of nesting so that the between-professional variance can be estimated. In the design phase of such trials not only the number of subjects needs to be derived but also the number of health professionals.
In this paper the optimal allocation was found such that an optimality criterion was minimized for a given budget. In other words, the optimal design was based on efficiency. It is also possible to use another operating characteristic to derive the optimal allocation, namely the statistical power. Given user-specified values of the effect sizes and a desired power level, this means finding the optimal allocation such that either the costs or total sample size are minimized. The optimal allocation thus obtained may not necessarily be the same as the one based on efficiency. Future research may focus on optimal allocations based on statistical power, thereby building on existing work for the between-subject design with homogeneous costs and variances [19]. As multiple placebo-treatment comparisons are made, various characterizations of power can be made, depending on whether the hypothesis tests for these contrasts are considered individually of simultaneously. In the latter case the power is defined for rejecting at least one hypothesis or rejecting all hypotheses. It would be interesting to derive optimal allocations based on these various characterizations of power for the incomplete within-subject with heterogeneous costs and variances.
Thus far the literature on optimal allocations for placebo-treatment comparisons in the incomplete within-subject design was scarce. This contribution provides methodology along with a Shiny app for trials with a placebo and two treatments. I hope applied researchers will benefit from it and be able to derive optimal allocations for their trial so that they can compare placebo to treatments in an efficient way.